The present invention relates to an image processing device and method and a recording medium, and particularly to an image processing device and method and a recording medium which enable correction of the influence of a light source on a measuring target object.
A technique for joining adjacent images of a number of so-called two-dimensional images (ordinary photographs or the like) is called mosaic processing. Generally, when the mosaic processing using a number of images is carried out, the overall densities and color tones of the respective images are made as coincident with one another as possible in advance. After that, in order to prevent the change of the color tone from becoming discontinuous in the boundary portion where the adjacent images are merged, blending of the density around the merging point is carried out.
There are several methods for making the overall densities and color tones of the respective images coincident with one another. As a method for correcting the difference in density and color tone, for example, there is a method called histogram coincidence method for converting the color tone of one image so that the histograms of the adjacent images in the overlap portion coincide with each other as much as possible, or a method called linear density converting method for finding the linear relation by a least square method and converting the color tone, which is a correction method on the assumption that linear conversion is realized between the pixels in the overlap portion. These methods are described in detail, for example, in the “Image Processing Standard Textbook” of the Image Information Education Promotion Society.
For the blending near the merging point, an α-blending method or the like is used for that the density of data in the overlap portion. In this method, the density is proportionally distributed over a predetermined section on both sides of the merging point.
In the α-blending, for example, the blending width is changed depending on the difference in density, or blending is carried out for each frequency band. These techniques are described in detail, for example, in Peter J. Burt and Edward H. Adelson, “A Multiresolution Spline with Application to Image Mosaics,” ACM Transactions on Graphics, Vol.2, No.4, October 1983, pages 217–236.
Meanwhile, in the case where the histogram coincidence method or the linear density converting method is used for textures (images) obtained by three-dimensional sensing, the unnaturalness of the merging of color tones in a seam part when merging textures cannot be eliminated because the relation between the object and the lighting is not considered at all.
In the α-blending method, the difference in color tone near the merging part can be eliminated to a certain extent. However, it cannot cope with the shooting of a moving target object with a fixed measuring device and lighting. That is, while the merging of color tones in the seam part can be corrected, the unevenness due to the lighting cannot be corrected.
Thus, apart from the mosaic processing for a two-dimensional image as a target, there is an image processing method for reproducing a three-dimensional model as an image, for example, a method called model-based rendering (MBR) or image-based rendering (IBR). This IBR enables generation of a realistic image by directly using a real image which is obtained by actually shooting a target object.
However, since most IBR techniques handle only geometric changes of the sight accompanied by changes of the visual point, which are essentially different from changes of the sight accompanied by changes of the light source, it is difficult to directly apply the conventional mosaic processing for a two-dimensional image.
Thus, there is a method called photometric image-based rendering, in which a number of real images having different light source directions are inputted and appropriately combined to generate an image of an arbitrary light source direction. This method is described in detail, for example, in Sadahiko Mitsuhashi, Hajime Miyaki, Yasuhiro Mukaikawa, and Ken Shakunaga, “Photometric Image-Based Rendering for Generation of Image of Arbitrary Light Source,” Shingaku Giho, PRMU-98-125, November 1998, pp. 17–24.
In this method, however, a number of light sources cannot be used. Therefore, it is difficult to prepare model data using a number of light sources.
As described above, in the conventionally proposed method, if a target object is rotated and shot from a number of angles, the light casting direction changes, thus making the shading on the target object uneven in the images picked up from the respective directions.
Other than the above-described methods, there is proposed a method for shooting the environment in which an object is shot, by a camera or the like, and reproducing the light source completely. This method is described in detail, for example, in Imari Sato, Yoichi Sato, and Katsuhumi Ikeuchi, “Superimposition of a virtual object onto a real image in consideration of optical consistency,” the Third Intelligence Information Media Symposium, December 1997, 23–32.
However, this method requires expensive parts such as a fish-eye lens and a CCD (charge coupled device) camera.